Cmput 551: Analyzing Abstraction and Approximation within Mdp/pomdp Environment
نویسندگان
چکیده
Markov Decision Process (MDP) has been used as a theoretical framework to solve AI problems for many decades. However, thus far most of the results cannot be effectively applied to most real world domains, which have large state spaces, indeterminant (fuzzy) goal states, incorrect guiding functions, and partial observability in general. This paper explores abstractions, approximations that occur in most real world domains and their effects on the performance of the agent within such an environment.
منابع مشابه
Analyzing Abstraction and Approximation within Mdp/pomdp Environment
Markov Decision Process (MDP) has been used as a theoretical framework to solve AI problems for many decades. However, thus far most of the results cannot be effectively applied to most real world domains, which have large state spaces, indeterminant (fuzzy) goal states, incorrect guiding functions, and in general are littered with occlusions and inaccuracies. This paper explores abstractions, ...
متن کاملTwo Heuristics for Solving POMDPs Having a Delayed Need to Observe
A common heuristic for solving Partial ly Observable Markov Decision Problems POMDPs is to rst solve the underlying Markov Decision Process MDP and then con struct a POMDP policy by performing a xed depth lookahead search in the POMDP and evaluating the leaf nodes using the MDP value function A problem with this approximation is that it does not account for the need to choose actions in order t...
متن کاملA POMDP Approximation Algorithm That Anticipates the Need to Observe
This paper introduces the even odd POMDP an approxi mation to POMDPs Partially Observable Markov Decision Problems in which the world is assumed to be fully observable every other time step This approximation works well for problems with a delayed need to observe The even odd POMDP can be converted into an equivalent MDP the MDP whose value function V MDP can be combined online with a step loo...
متن کاملA POMDP Approximation Algorithm thatAnticipates the Need
This paper introduces the even-odd POMDP, an approximation to POMDPs (Partially Observable Markov Decision Problems) in which the world is assumed to be fully observable every other time step. This approximation works well for problems with a delayed need to observe. The even-odd POMDP can be converted into an equivalent MDP, the 2MDP, whose value function, V 2MDP , can be combined online with ...
متن کاملHindsight is Only 50/50: Unsuitability of MDP based Approximate POMDP Solvers for Multi-resolution Information Gathering
Partially Observable Markov Decision Processes (POMDPs) offer an elegant framework to model sequential decision making in uncertain environments. Solving POMDPs online is an active area of research and given the size of real-world problems approximate solvers are used. Recently, a few approaches have been suggested for solving POMDPs by using MDP solvers in conjunction with imitation learning. ...
متن کامل